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Over  the  last  decade,  the  President  and  Congress  have  tasked  the  Department  of  Defense 
(DoD)  with  reducing  expenditures  and  finding  areas  for  cost  savings.  DoD  officials  have 
responded  with  initiatives  such  as  Better  Buying  Power,  Should  Cost  and  Bending  the  Cost 
Curve.  Senior  leaders  are  debating  which  functional  areas  need  to  be  cut  given  the  reduced 
budget.  What  if  we  could  keep  our  core  functions,  while  simultaneously  reducing  costs?  This 
paper  purports  that  one  avenue  to  accomplish  this  is  to  overhaul  the  overly  rigid,  bureaucratic 
acquisitions  process.  The  new  acquisitions  guidance,  DoDI  5000.02,  represents  a  step  in  the  right 
direction  by  allowing  for  an  incremental  delivery  process,  but  implementation  of  Agile  principles 
are  lacking.  In  these  times  of  fiscal  austerity,  the  DoD  must  focus  on  Agile  implementation  in 
three  main  areas:  training,  business  process  re-engineering  and  contracting  guidance. 

It  is  beyond  the  scope  of  this  paper  to  identify  a  solution  that  will  positively  affect  the 
vast  array  of  acquisitions  projects.  In  order  to  offer  specific  recommendations  and  bound  the 
topic  of  research,  the  focus  of  this  paper  is  on  information  technology  (IT)  acquisitions 
specifically,  which  includes  business  systems,  command  and  control,  computing  and 
communications  infrastructure,  Intelligence  Surveillance  and  Reconnaissance  (ISR),  and  space 
and  cyber  weapon  systems.  The  rationale  for  this  focus  is  that,  “at  the  core  of  the  ability  to 
achieve  integration  and  maintain  agility  is  the  ability  of  the  DoD  to  produce  and  evolve 
software”  (Critical  Code:  Software  Producibility  for  Defense,  2010).  In  1960,  software 
accounted  for  8%  of  the  functionality  in  the  F-4,  compared  with  80%  of  the  functionality  in  the 
F-22  (Critical  Code:  Software  Producibility  for  Defense,  2010).  As  Figure  1  below  illustrates,  the 
amount  of  software  embedded  in  weapons  systems  is  continuing  to  increase.  As  best  stated  by  an 
Air  Force  general  officer,  “The  B-52  lived  and  died  by  the  quality  of  its  sheet  metal.  Today,  our 
aircraft  will  live  or  die  by  the  quality  of  our  software”  (Hagan,  Hurt,  &  Sorenson,  2).  The  F-35  is 
currently  DoD’s  most  costly  program  and  it  is  experiencing  critical  software  issues.  Eighty 
percent  of  the  issues  identified  by  the  Autonomic  Logistics  Information  System  (ALIS),  which 
determines  potential  maintenance  problems,  are  false  positives  (Everstine,  2015).  These  examples 
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illustrate  software’s  critical  role  in  the  weapons  of  the  future.  The  change  in  methodology  this 
paper  elaborates  upon  can  also  be  applied  to  areas  outside  of  software,  but  this  has  been  the 
primary  area  of  applicability.  It  is  important  to  demonstrate  successful  application  of  the  below 
principles  in  software  development  to  make  a  business  case  for  extending  these  principles  to 
other  areas  like  manufacturing. 
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Figure  1:  Growth  in  Software  Lines  of  Code  (SLOC)  in  Avionics  Software  (Hagan,  C.,  Hurt,  S., 

&  Sorenson,  J,  2012) 

At  first  glance,  the  acquisitions  field  might  not  seem  like  the  most  obvious  place  to 
maximize  value  or  create  cost  savings  for  the  Air  Force.  Comparatively  speaking,  procurement, 
the  area  most  often  associated  with  acquisitions,  is  only  the  third  highest  expenditure  in  the  Air 
Force’s  portion  of  the  FY16  Defense  Budget,  falling  behind  Operations  and  Maintenance,  and 
Military  Personnel  costs  (SAF/FMB,  2015).  However,  the  lifecycle-centric  nature  of  the 
acquisitions  enterprise  means  that  funding  for  an  acquisition  can  come  from  research, 
development,  test  and  evaluation,  procurement,  or  operation  and  maintenance  funds.  When  one 
takes  this  into  account,  acquisitions  potentially  affects  75%  of  the  Air  Force’s  projected  FY16 
budget,  or  up  to  $91  billion  dollars. 

The  last  decade  has  seen  numerous  efforts  by  the  Air  Force,  the  DoD,  and  Congress  to 
reform  how  agencies  implement  the  acquisitions  process  because  of  a  perception  that  the  system 
is  broken.  Unfortunately,  this  is  not  a  new  trend.  According  to  one  count  by  former  Secretary  of 
Defense  Donald  Rumsfeld,  between  1975  and  2001  there  were  “128  different  studies  and  reports 
on  reforming  the  [acquisitions]  system”  (Spring,  2005).  Certainly,  the  last  fifteen  years  yielded 
significant  evidence  that  the  acquisitions  process  is  far  from  efficient.  Between  2001  and  2011, 
the  DoD  spent  $46  billion  dollars  on  twelve  weapons  systems  that  it  ultimately  cancelled  with  no 
tangible  result  (Reed,  2011).  According  to  the  GAO,  the  98  Major  Defense  Acquisitions 
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Programs  from  FY10  were  collectively  $406  billion  dollars  over  budget  and  averaged  22  months 
behind  schedule  since  their  first  full  estimate  (Hofbauer,  Sanders,  Ellman,  &  Morrow,  2011).  In 
short,  the  DoD  spends  a  large  amount  of  money  for  projects  that  are  either  not  delivered  or  are 
delivered  far  over  budget  and  significantly  behind  schedule. 

What  drives  this  deficiency?  Why  can’t  the  acquisitions  process  deliver  the  required 
products  on  time  and  on  budget?  One  serious  issue  might  be  the  acquisitions  process  itself.  For 
the  last  twenty  plus  years,  the  Air  Force  has  used  the  waterfall  approach  for  software 
development  as  illustrated  in  Figure  2  below.  This  approach  is  very  linear  in  nature  and  relies  on 
several  key  assumptions,  which  include  well-defined  requirements  and  changes  to  the 
requirements  will  be  very  limited  and  small  in  nature  (Leffingwell,  2007).  Given  the  rapid  rate  at 
which  technology  advances,  these  are  no  longer  valid  assumptions  upon  which  the  Air  Force  and 
its  acquisitions  programs  can  rely.  Any  changes  made  to  the  requirements,  even  those  that  are 
realized  to  be  obsolete  based  on  new  information,  must  go  through  a  very  rigorous  engineering 
change  proposal  (ECP)  process.  The  sheer  amount  of  documentation  required  in  this  process 
causes  schedule  delays  and  is  accompanied  by  a  hefty  price  tag.  An  estimate  from  1992  showed 
that  over  a  six  month  period,  one  program  cost  $22  million  dollars  and  more  than  325,000  man 
hours  to  deal  with  the  bureaucratic  requirements  (Spring,  2005).  The  bottom  line  is  that  this 
model  is  an  anachronism;  this  process  is  wholly  ineffective  for  an  age  of  rapidly  changing 
technologies  and  a  very  dynamic  threat  environment.  Using  this  process,  it  takes  an  average  of 
8 1  months  to  develop  new  software  for  the  Air  Force,  while  most  private  companies  plan  on  a 
12-18  month  cycle  to  deliver  new  software  to  put  them  within  the  two  year  cycle  of  technology 
development  dictated  by  Moore’s  Law  (Modigliani  &  Chang,  2014). 


Figure  2:  Waterfall  Process  (Palmquist,  Lapham,  Miller,  Chick,  &  Ozkaya,  2013) 

The  solution  to  this  problem  is  a  new  process  for  acquisitions.  The  2015  updates  to  the 
DoD  acquisitions  guidance,  DoDI  5000.02,  recognized  the  need  for  an  alternative  to  the  linear 
software  development  model  illustrated  above.  The  guidance  included  a  model  that  allows  for 
incremental  software  development,  but  does  not  specifically  mention  Agile  within  the  document. 
It  is  important  to  note  that  incremental  development  does  not  necessarily  mean  that  an 
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organization  is  implementing  Agile  principles,  which  will  be  elucidated  below.  The  next  step  is 
for  the  Air  Force  to  take  the  implementation  steps  needed  to  ensure  that  an  environment  exists  in 
which  Agile  can  succeed.  At  a  minimum,  the  Air  Force  needs  a  new  approach  to  software 
acquisitions  in  order  to  maximize  functionality  of  new  systems,  while  simultaneously  keeping 
our  existing  systems  technology  up  to  date.  This  system  needs  to  make  requirement  prioritization 
and  regular  contact  between  contractor  and  the  end  user  a  priority,  which  minimizes 
requirements  creep  by  forcing  the  customer  to  stay  focused  on  the  most  important  issues  while 
keeping  them  aware  of  limitations.  It  could  also  reduce  the  need  for  excessive  bureaucracy  and 
paperwork  requirements  by  utilizing  the  customer/contractor  interaction  to  fulfill  the  same 
documentation  purpose.  Finally,  the  new  process  needs  to  be  scalable  and  potentially  applicable 
to  the  realm  of  hardware  as  well.  The  good  news  is  there  is  a  process  that  fulfills  all  of  these 
requirements,  and  the  process  is  Agile  Development. 

A  New  Model:  Agile  Development 

As  described  above,  a  rigid  acquisitions  process  cannot  support  the  fluid  contingency 
operations  in  which  we  operate.  The  next  question  is,  why  Agile?  Adding  flexibility  and 
increasing  capabilities  should  be  the  goal,  not  ditching  both.  The  Air  Force’s  30-year  strategy 
mentions  agility  twenty-one  times  in  its  twenty  pages,  with  Secretary  James  specifically  citing 
agility  as  a  way  of  “breaking  paradigms  and  leveraging  our  technology”,  especially  through  how 
“we  acquire  and  field  weapons  systems”  (U.S.  Air  Force,  2014).  With  Agile,  not  only  do  we 
build  flexibility  demanded  from  senior  Air  Force  leadership,  but  we  also  obtain  finite 
improvements  and  results  from  the  Agile  method. 

The  Agile  Development  process  began  a  few  decades  ago  with  the  growing  need  to  keep 
pace  with  the  rapidly  increasing  software  development  process.  Moore’s  Law,  developed  by 
Gordon  Moore  in  the  1970’s,  brought  to  realization  that  computing  power,  and  the  technologies 
associated  with  it,  would  double  every  eighteen  to  twenty-four  months.  This  exponential  increase 
in  software  requires  an  acquisitions  process  that  finds  the  ability  to  match  that  evolutionary 
speed.  In  Agile,  we  find  that  match.  The  core  tenets  of  Agile  include  focusing  on  small,  frequent 
capability  releases;  valuing  working  software  over  comprehensive  documentation;  responding 
rapidly  to  changes  in  operations,  technology,  and  budgets;  and  actively  involving  the  users 
throughout  development  to  ensure  high  operational  value  (Magliani  and  Chang,  2014).  The  Agile 
life-cycle  exhibits  a  circular  approach  in  which  development  stages  begin  and  are  tested  at  the 
end  of  each  stage,  or  iteration  as  illustrated  in  the  Figure  3  below.  The  Agile  Methodology 
website  notes  that  “in  an  agile  paradigm,  every  aspect  of  development  -  requirements,  design, 
etc.  -  is  continually  revisited  throughout  the  lifecycle”  (Agile  Methodology,  2008).  Continually 
revisiting  requirements  throughout  development  allows  more  flexibility  and  adaptability  to 
changes  as  they  arise. 
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Figure  3:  High  Level  Agile  Process  (Gorans  &  Kruchten,  2014) 

The  Agile  approach  to  software  development  is  a  set  of  principles.  The  IT  industry  has 
developed  multiple  Agile  methods  that  are  tailored  to  the  specific  needs  of  different  types  of  IT 
projects.  It  is  outside  the  scope  of  this  paper  to  describe  the  numerous  methodologies  in  depth, 
given  that  the  focus  is  on  implementing  an  environment  in  which  any  of  the  Agile  processes  can 
succeed  if  implemented  properly.  It  is  important  to  note  that  there  are  methodologies  for  smaller 
efforts  in  addition  to  scalable  models  for  larger,  more  complex  efforts.  These  earlier 
methodologies  include,  but  are  not  limited  to,  Scrum,  Kanban,  Extreme  Programming  and 
Crystal.  Although  some  of  the  methodologies  are  ‘heavier’  when  compared  to  others,  all  of  these 
methods  are  extremely  ‘lightweight’  when  compared  to  the  documentation  that  is  required  in  the 
traditional  waterfall  approach. 

Specific  models  for  enterprise  level  efforts  include  Scaled  Agile  Framework,  Disciplined 
Agile  Delivery,  Large  Scale  Scrum  and  Scaled  Scrum  (Mike  Cottmeyer,  personal 
communication,  August  31,  2015).  This  is  not  meant  to  be  an  all-inclusive  list  of  Agile 
methodologies  and  novel  approaches  may  be  developed  based  on  the  rapidly  evolving  nature  of 
software  development.  The  scalable  models  conform  with  the  same  principles  as  the  earlier 
models,  but  they  illustrate  how  to  integrate  multiple  teams  depending  upon  the  size  of  the  project 
and  provide  a  portfolio  view.  In  order  to  demonstrate  the  differing  levels  of  effort,  Figure  4 
below  illustrates  a  typical  Scrum  process,  whereas  Figure  5  offers  a  Scaled  Agile  Framework 
model. 
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Figure  4:  Overview  of  Scrum  Process  (Glaiel,  2012) 
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Figure  5:  Scaled  Agile  Framework  Model  (Wrubel ,  Miller,  Lapham,  &  Chick,  2014) 

For  the  purposes  of  this  analysis,  the  focus  will  be  on  implementation  of  Agile  principles 
in  software  development.  However,  Agile  methodologies  can  be  used  to  improve  efficiency  in 
other  industries.  One  example  is  Toyota  Motor  Corporation’s  automobile  production  line,  which 
uses  a  system  called  Kanban,  literally  translated  to  “signboard.”  “Just-in-Time”  was  the  original 
idea.  Promoted  by  Toyota  vice-president  Taiichi  Ohno,  he  called  for  the  use  of  Kanban  to  ensure 
Toyota  produced  “only  what  is  needed,  when  it  is  needed,  and  in  the  amount  needed”  (Toyota, 
2015).  Inspired  by  the  supply  chain  at  supermarkets,  Kanban  are  slips  of  paper,  barcodes,  or 
other  means  of  tracking  inventory.  Through  carefully  tracking  and  managing  their  supply  chain 
throughout  the  production  cycle,  Toyota  is  able  to  eliminate  overproduction  by  only  restocking 
production  parts  that  have  been  used.  By  keeping  local  inventory  small,  Toyota  production 
plants  are  able  to  keep  a  wider  variety  of  parts  on  hand,  which  leads  to  the  ability  to  produce 
vehicles  with  different  specifications  at  any  time.  When  parts  are  used  in  production,  a  Kanban 
slip  informs  the  supply  chain  that  more  of  those  parts  are  needed  at  the  production  plant  (Toyota, 
2015).  The  end  result  of  this  process  is  that  Toyota  is  able  to  efficiently  produce  more 
automobiles,  with  less  parts,  and  respond  with  agility  to  changes  in  market  demand.  This 
example  illustrates  the  success  of  Agile  within  the  manufacturing  industry.  More  research  is 
needed  to  detennine  if  a  similar  model  can  be  used  for  non-software-centric  acquisitions  in  the 
DoD. 


Reductions  in  cost  through  continuous  improvement  allows  for  a  greater  return  on  our 
investment.  A  group  of  researchers  from  MIT  found  that  using  Agile  in  software  development 
can  lead  to  anywhere  from  “5%  to  61%  reductions  in  cost,  24%  to  58%  reductions  in 
development  time,  and  1 1%  to  83%  reductions  in  product  defects”  (Glaeil,  Madnick,  and 
Mouton,  2013).  According  to  a  201 1  report  from  the  Standish  Group,  Agile  projects  are  three 
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times  more  successful  than  traditional  projects  (Cohn,  2011).  A  survey  on  the  literature  of 
Agile’s  return  on  investment  (ROI),  as  shown  in  Figure  6  below,  further  illustrates  the  numerous 
benefits  of  this  process.  These  reductions  in  cost,  coupled  with  a  proven  return  on  investments, 
shows  that  using  the  Agile  method  not  only  works  but  is  a  viable  option  going  forward.  Thus  far, 
we  have  identified  the  inadequacies  of  the  current  waterfall  process  and  made  a  business  case  for 
Agile  adoption.  Next,  we  will  elaborate  upon  the  fundamentals  of  Agile  and  the  different 
methodologies  that  conform  to  Agile  principles. 


Source 

Findings 

Responses 

1998 

Harvard 

(Thomke  et  at,  1998) 

50%  reduction  in  engineering  effort 

55%  improvement  in  time  to  market 

925%  improvement  in  number  of  changes  allowed 

391 

1998 

Harvard 

(MacCormack,  1998) 

48%  productivity  increase  over  traditional  methods 

38%  higher  quality  associated  with  more  design  effort 

50%  higher  quality  associated  with  iterative  development 

29 

1999 

Boston  College 
(Fichman  et  at,  1999) 

38%  reduction  in  time  to  produce  working  software 

50%  time  to  market  improvement 

50%  more  capabilities  delivered  to  customers 

28 

2003 

Reifer  Consultants 
(Reifer,  2003) 

20%  reported  productivity  gains 

10%  reported  cost  reductions 

53%  reported  time-to-market  improvements 

78 

2003 

Shine  Technologies 
(Johnson,  2003) 

49%  experienced  cost  reductions 

93%  experienced  productivity  increases 

88%  experienced  customer  satisfaction  improvements 

131 

2004 

CIO  Magazine 
(Prewitt,  2004) 

28%  had  been  using  agile  methods  since  2001 

85%  initiated  enterprise-wide  agile  methods  initiatives 

43%  used  agile  methods  to  improve  growth  and  marketshare 

100 

2006 

Digital  Focus 
(Digital  Focus,  2006) 

27%  of  software  projects  used  agile  methods 

23%  had  enterprise-wide  agile  methods  initiatives 

51%  used  agile  methods  to  speed-up  development 

136 

2006 

Version  One 
(Version  One,  2006) 

86%  reported  time-to-market  improvements 

87%  reported  productivity  improvements 

92%  reported  ability  to  dynamically  change  priorities 

722 

2006 

AmbySoft 
(Ambler,  2006) 

4 1  %  of  organizations  used  agile  methods 

44%  reported  improved  productivity,  quality,  and  costs 

38%  reported  improvements  in  customer  satisfaction  levels 

4,232 

2007 

AmbySoft 
(Ambler,  2007) 

69%  of  organizations  had  adopted  agile  methods 

89%  of  agile  projects  had  a  success  rate  of  50%  or  greater 

99%  of  organizations  are  now  using  iterative  development 

781 

2007 

UMUC 
(Rico,  2007) 

70%  of  developers  using  most  aspects  of  agile  methods 

26%  of  developers  had  improvements  of  50%  or  greater 

Agile  methods  are  linked  to  improved  website  quality 

250 

Figure  6:  Literature  Review  of  Agile  ROI  (Rico,  2015) 

Examples  of  Agile  within  DoD 

Acquisitions  in  the  private  sector  are  inherently  different  than  government  acquisitions 
due  to  issues  such  as  funding  stream,  regulations,  and  documentation  requirements,  among 
others.  Thus,  the  next  question  posed  was  whether  or  not  Air  Force  programs  are  executing  Agile 
today.  Market  research  and  discussions  with  other  program  managers  revealed  that  Agile  is  being 
conducted  by  at  least  five  program  management  offices  (PMOs)  in  three  different  MAJCOMs. 
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The  team  interviewed  two  PMOs  to  offer  examples  of  Agile  success  stories,  which  are  provided 
below. 


Two  programs  using  Agile  methodologies  appeared  in  Special  Operations  Command 
(SOCOM).  One  example  is  the  Lead-Off-Hitter  (LOH)  program,  which  develops  software 
updates  for  the  MQ-9  Reaper.  Traditionally,  the  Air  Combat  Command  (ACC)  Operational 
Flight  Program  (OFP)  handled  software  development  for  the  MQ-9.  Delivery  of  requirements 
usually  occurred  after  four  to  five  years,  typically  coupled  with  cost  growth.  In  one  cycle,  the 
program  office  conducted  1 1  requirements  modifications  and  costs  grew  by  over  $12M  from 
2006  to  2013.  SOCOM  leadership  deemed  this  typical  cost  growth  and  late  delivery  schedule 
insufficient  to  meet  the  warfighters’  needs,  and  as  a  result  stood  up  the  LOH  program.  This 
program  office  operates  under  the  following  principles:  schedule  driven  releases  over  the 
traditional  content  releases,  limited  technical  scope  based  on  release  schedule,  regular  end-user 
participation,  and  require  documentation  when  necessary.  The  program  began  with  awarding  a 
two-year  cost  plus  fixed  fee  contract  for  $16M,  and  operates  on  a  six-month  delivery  cycle. 
Although  requirements  changes,  to  include  additions  and  deletions,  occur  on  a  weekly  basis,  the 
program  office  has  experienced  zero  cost  growth.  Requirements  are  simply  prioritized,  with  no 
contract  modification  needed,  based  on  the  need  of  the  end-user.  From  February  2014  to  June 
2015,  the  program  office  delivered  four  releases  that  are  supporting  over  800  requirements  and 
are  working  an  additional  1,000  requirements.  The  team  also  fixed  182  software  defects  and 
resolved  92  known  anomalies  (Jamieson  Pierce,  personal  communication,  August  14,  2015). 

Another  example  within  SOCOM  is  the  MALET  JSIL  Aircrew  Training  Device  (M- 
JADT)  team.  The  original  MQ-9  trainer  is  a  plug-and-play  training  device  developed  by  the 
Anny  to  address  a  capability  gap  in  Reaper  training.  Using  Agile,  AFSOC  built,  tested,  and 
fielded  an  MQ-9  variant  based  on  the  Army’s  MQ-1C  trainer  in  five  months.  The  team 
accomplished  this  feat  with  an  85%  reduction  in  cost  and  schedule  compared  to  the  original 
plug-and-play  trainer  developed  by  the  Army.  Each  AFSOC  trainer  is  $70K  and  can  be  delivered 
60  days  after  initial  order.  Six  trainers  have  been  fielded,  with  an  additional  fourteen  more  being 
made  (Jamieson  Pierce,  personal  communication,  August  14,  2015). 

Given  that  SOCOM  operates  within  different  acquisition  regulations,  it  was  important  for 
the  team  to  find  Agile  examples  outside  of  the  special  operations  community.  The  Patriot 
Excalibur  (PEX)  program  in  Air  Force  Material  Command  has  been  executing  Agile,  specifically 
Extreme  Programming  (XP)  with  a  Scrum  management  philosophy,  since  2003.  The  program 
automates  squadron  processes  in  three  areas:  scheduling,  training,  and  standards  and  evaluation. 
Squadrons  have  voluntarily  adopted  the  program  and  it  now  serves  over  700  AF  and  Air 
National  Guard  squadrons.  Prior  to  Agile  adoption,  PEX  operated  on  an  18  month  delivery  cycle 
and  received  low  customer  satisfaction  (Garcia-Miller,  Lapham,  Boston,  Sharp,  &  Goshom, 
2011).  The  development  team  suggested  the  program  office  switch  to  an  Agile  process  given  the 
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current  schedule  and  satisfaction  issues.  The  result  is  a  22-week  release  schedule,  that  on  average 
addresses  480  requirements  and  10  software  defects  (Johnson,  Cheng,  Misiazek,  Greytak,  & 
Boston,  2011).  The  program  office  attributed  success  to  three  factors,  besides  those  inherent  in 
XP  and  Scrum.  The  first  is  adequate  Agile  training  in  the  program  office  by  professionals.  In 
order  to  ensure  success,  the  program  halted  work  to  focus  on  training  and  hired  consultants. 
Second,  continuous  engagement  with  the  end  user  is  absolutely  vital.  PEX  hosts  an  annual  user 
forum,  attended  by  over  200  users  in  201 1,  to  obtain  new  user  stories  and  solicit  feedback  on  the 
current  application.  User  interaction  is  more  than  just  an  annual  occurrence;  subject  matter 
experts  (SMEs)  reside  within  the  PMO  and  work  alongside  the  development  team.  The  last,  and 
one  of  the  most  important  success  factors  mentioned,  is  a  flexible  contract  vehicle  (Garcia- 
Miller,  Lapham,  Boston,  Sharp,  &  Goshom,  2011).  Given  the  flexible  nature  of  Agile  user 
stories  and  constant  reprioritization,  it  is  critical  to  have  a  contract  strategy  that  is  flexible  and 
will  not  require  a  modification  with  each  change.  The  PEX  PMO  used  a  cost  plus  incentive  fee 
time  and  materials  contract.  Kelly  Goshom,  the  PEX  PM  for  over  16  years,  states  this  was  a  key 
success  factor.  The  time  and  materials  contract  gave  her  team  the  flexibility  needed  when  it  came 
to  user  story  development  that  is  not  available  with  either  fixed-price  or  cost  reimbursable 
contracts  (Kelly  Goshorn,  personal  communication,  August  17,  2015).  The  PEX  PMO  received 
several  awards  in  recognition  for  its  continued  success,  to  include,  the  2003  Crosstalk  award  for 
The  Top  5  Quality  Software  Programs  in  the  U.S.  government  and  a  finalist  for  the  2008  AF 
Chief  of  Staff  Team  Excellence  Award  (Garcia-Miller,  Lapham,  Boston,  Sharp,  &  Goshom, 
2011). 

The  F-22  program  office  is  not  currently  conducting  a  complete  Agile  effort,  but  an 
Iterative  model  that  applies  some  agile  tenets.  They  are  moving  to  the  Scaled  Agile  Framework 
(SAFe)  for  a  software  modernization  effort.  An  avionics  engineer  mentioned  that  conducting  a 
project  of  their  magnitude  with  a  traditional  agile  approach  would  be  extremely  difficult  and 
having  a  scalable  model,  like  SAFe,  is  vital.  The  team  has  noticed  several  benefits  with  the  agile 
aspects  of  the  Iterative  methodology,  although  it  is  too  early  to  classify  the  implementation  as 
successful.  Early  testing  of  the  software  has  led  to  the  discovery  of  defects  that  would  have 
caused  major  setbacks  if  found  at  the  end  of  development,  per  the  waterfall  methodology.  Agile 
metrics  are  another  tool  the  program  office  found  extremely  beneficial.  Metrics  included,  defect 
tracking,  task  completion  and  the  speed  at  which  the  development  team  completed  the  tasks,  to 
name  a  few.  These  metrics  provided  early  insight  into  possible  schedule  issues  and  allowed  for 
adjustment  early  in  the  process  (Bradley  Bernard,  personal  communication,  August  17,  2015). 
The  examples  cited  above  illustrate  that  even  with  the  bureaucratic  DoD  processes,  program 
offices  are  successfully  implementing  Agile  principles. 

The  question  becomes,  how  can  the  Air  Force  emulate  these  successes  on  a  much  larger 
scale?  As  emphasized  by  Mr.  Jamieson  Pierce  from  the  Lead-Off-Hitter  program,  Agile  is  not 
the  silver  bullet  that  will  solve  all  of  the  Air  Force’s  acquisitions  issues.  It  would  also  be 
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incorrect  to  assume  that  anyone  can  manage  an  Agile  project  successfully.  A  GAO  report 
identified  14  factors  that  are  inhibiting  Agile  adoption  and  application  in  DoD  (Government 
Accountability  Office  Effective,  2015).  Generally  speaking,  these  factors  can  all  be  tied  back  to 
trying  to  implement  Agile  by  simply  adding  to  current  processes  and  models  of  operation,  rather 
than  understanding  that  Agile  requires  a  completely  new  process  in  and  of  itself  (Mueller,  2). 
The  next  section  of  this  report  addresses  the  steps  that  the  Air  Force  must  take  in  order  to  ensure 
successful  implementation  of  Agile  on  a  wide  scale,  which  includes  training,  business  process 
updates,  and  contracting  guidance. 


Training 

An  SEI  report  on  considerations  for  Agile  implementation  within  the  DoD  identified 
training  as  a  foundational  concern  (Lapham,  Considerations  for  Using  Agile  in  DoD  Acquisition, 
2010).  Changing  from  the  traditional  acquisition  processes  to  Agile  methodologies  will  force 
some  culture  shock,  as  well  as  some  organizational  resistance.  A  source  of  this  shock  and 
resistance  is  simply  a  reaction  to  the  unknown.  Training  and  familiarization  will  smooth  the 
transition  to  Agile  while  building  the  skillsets  personnel  need  to  succeed. 

The  first  step  in  implementing  Agile  methodologies  in  the  Air  Force  is  to  address  how 
personnel  are  trained.  The  primary  training  focus  would  be  for  the  broader  acquisition 
community,  to  include  the  program  manager,  contracting  officer,  engineering,  and  the  user 
community,  among  others.  Simply  training  the  program  manager  is  insufficient,  because  Agile 
requires  buy-in  from  the  entire  government  team.  As  subsequent  sections  will  demonstrate,  a 
contracting  officer’s  understanding  of  Agile  methodologies  is  needed  in  order  to  execute  a 
flexible  contracting  strategy  that  supports  Agile,  while  engineering  expertise  can  be  used  to 
develop  a  business  process  more  conducive  to  Agile.  It  is  imperative  that  senior  leaders  also 
receive  training  because  their  buy-in  is  often  critical  for  program  success.  Currently,  the  Air 
Force  acquisition  workforce  is  trained  by  the  Defense  Acquisition  University  (DAU),  and  the 
vast  majority  of  the  courses  are  delivered  through  online  courseware  (DAU,  2013).  Despite  the 
widespread  commercial  usage  of  Agile  principles,  DAU’s  list  of  182  courses  does  not  contain  a 
single  course  focusing  on  Agile  methodologies  for  program  management  (DAU,  2015). 

There  are  three  courses  of  action  (CO As)  the  Air  Force  could  pursue  to  ensure 
acquisitions  officers  are  properly  equipped  to  run  programs  utilizing  Agile  methodologies.  First, 
DAU  could  develop  courseware  that  prepares  acquisitions  officers  to  manage  programs  that 
utilize  one  or  more  of  the  Agile  methodologies.  Second,  DAU  could  transform  itself  to  deliver  an 
accredited  Master’s  degree  in  Program  Management.  Third,  the  Air  Force  could  pay  an 
established  private  institution  to  deliver  the  training.  These  COAs  are  not  necessarily  mutually 
exclusive.  The  first  two  COAs  are  long  term  solutions  needed  in  order  to  implement  Agile,  while 
accounting  for  unique  Air  Force  requirements.  The  third  option  is  the  optimal  solution  to  assist 
programs  in  the  short  term. 
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The  private  sector  has  increasingly  been  using  Agile  to  quickly  and  efficiently  produce 
software  for  their  customers  (Northern,  Mayfield,  Benito,  &  Casagni,  2010).  Agile  training  has 
been  in  high  demand  across  the  IT  industry  and  has  since  migrated  into  other  industries.  Today, 
there  are  numerous  companies  and  intuitions  offering  coaching,  classes,  or  certifications  in  Agile 
methodologies.  The  Project  Management  Institute  (PMI)  is  the  premier  institution  for  project 
management  professionals,  and  they  offer  2-  or  3-day  courses  on  Agile,  at  a  cost  of  $1,545  or 
$2,275,  respectively  (PMI,  2015).  An  additional  $495  would  enable  an  acquisitions  professional 
to  test  for  a  certification  from  PMI  in  Agile  practices.  Acquisitions  officers  would  then  be  able 
to  certify,  similar  to  the  current  certifications  obtained  through  DAU,  that  they  meet  a 
professional  standard  and  have  a  basic  understanding  of  Agile  methodologies  from  an  academic 
perspective.  Requiring  an  Agile  certification  for  program  offices  using  an  these  methodologies 
would  bring  the  career  field  in  line  with  the  customers  they  often  support  and  would  further 
professionalize  the  career  field.  During  the  last  10  years,  DoD  Information  Assurance  (IA) 
personnel  with  elevated  decision  authority,  or  elevated  access  to  Information  Systems,  were 
required  to  obtain  and  maintain  a  requisite  certification  in  the  fundamentals  of  IT  Security. 
Additional  members  of  the  DoD  performing  an  IA  role  may  be  required  to  maintain  certifications 
commensurate  with  their  responsibilities  (IASE,  2015).  It  can  be  argued  that  acquisitions 
professionals  who  manage  programs  with  large  dollar  values,  large  scope  of  impact  or  increased 
risks  associated  with  the  program,  should  be  required  to  maintain  a  professional  certification  in 
Agile  practices. 

Education  regarding  Agile  principles  from  an  academic  perspective  is  vital,  but  it  cannot 
be  supplanted  for  coaching  from  experienced  Agile  practitioners.  Numerous  companies  offer 
coaching  services  and  tools,  many  specializing  in  a  certain  Agile  methodology,  to  other 
businesses.  In  order  to  offer  operational  experience  to  the  Agile  workforce,  continuing  education 
could  utilize  Agile  coaches  or  mentors  to  run  quarterly  training  sessions  that  focus  on  applying 
certain  Agile  methodologies  to  specific  programs.  Although  quarterly  training  will  provide 
practical  examples,  it  cannot  serve  as  the  only  hands-on  training.  Coaches  and  mentors  should 
provide  one-on-one  training  to  individual  programs  due  to  the  unique  nature  of  each  program  and 
the  numerous  methodologies  available.  As  mentioned  by  one  program  office,  Assistance  and 
Advisory  Services  (A&AS)  contracts  offer  one  avenue  for  obtaining  this  type  of  expertise 
(Michael  Bigelow,  personal  interview,  September  1,  2015). 

These  same  coaches  and  mentors  should  be  utilized  to  bring  Agile  courseware  to  DAU. 
The  cost  of  adding  Agile  courseware  to  DAU  would  have  to  be  examined,  but  the  FY14  budget 
request  from  DAU  included  $19  million  for  curriculum  development  (DAU,  2013).  DAU  should 
prioritize  incorporation  of  Agile  courseware  into  future  development  plans.  Additionally, 
according  to  a  report  from  2010,  DAU  and  the  DoD  do  not  track  metrics  to  detennine  the  long¬ 
term  effectiveness  of  the  training  they  provide  (Government  Accountability  Office,  2010).  The 
GAO  recommended  the  implementation  of  such  metrics,  but  the  recommendation  was  not 
accepted  by  the  DoD.  As  part  of  the  non-concur  by  the  DoD,  it  was  indicated  that  metrics  in 
place  sufficiently  capture  the  effectiveness  of  DAU’s  training  (Government  Accountability 
Office,  2010).  However,  these  metrics  do  more  to  capture  DAU’s  capacity  to  support  the 
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acquisitions  career  field,  rather  than  the  actual  effectiveness  of  the  acquisitions  career  field.  The 
adoption  of  Agile  would  allow  for  effectiveness  metrics  to  be  analyzed  by  DAU,  DoD,  and 
GAO,  since  such  metrics  are  common  in  Agile  programs. 

Once  established,  DAU  could  move  to  become  an  industry  leader  by  developing  a  true 
Master’s  Program,  much  like  an  online  degree  offered  by  the  civilian  academic  community.  The 
program  could  be  structured  in  9-week  tenns.  Courseware  would  be  comprised  of  discussion 
boards,  collaborative  projects,  case  studies,  papers,  and  exams,  all  presented  and  proffered  by  a 
professor  in  the  field  of  study  related  to  the  courseware.  This  would  help  change  the  Air  Force’s 
and  the  DoD’s  views  of  acquisitions  from  an  assignment  or  career  field  to  a  true  profession.  As 
previously  stated,  Agile  is  not  a  one  size  fits  all  solution,  nor  does  it  keep  programs  immune 
from  failure.  Training  and  professional  Agile  coaches  and  mentors  are  a  critical  first  step  that 
will  lead  to  the  successful  implementation  of  Agile.  After  adequate  training  is  provided,  DoD 
must  ensure  the  business  processes  support  Agile  principles. 

Business  Process  Reengineering 

Training  is  a  necessary  first  step  to  ensure  that  the  acquisitions  community  and  support 
staff  is  familiarized  with  the  specific  Agile  methodology  being  proposed  by  the  contractor.  As 
mentioned  earlier,  the  new  DoDI  5000.02  allows  for  incremental  delivery,  but  the  Air  Force  has 
not  considered  the  differences  inherent  in  an  Agile  business  process.  The  waterfall  methodology 
relies  on  immense  amounts  of  documentation  to  track  and  report  progress.  Due  to  the  nature  of 
the  acquisition  phases  of  development,  testing,  and  delivery,  this  documentation  allowed  the  Air 
Force  to  report  progress  at  milestones  and  attempt  to  peek  behind  the  curtain.  It  is  incorrect  to 
assume  that  Agile  does  not  provide  any  documentation.  Relative  to  the  old  model,  Agile  focuses 
on  providing  just  enough  documentation  to  ensure  continuity  of  operations  and  success  of  the 
program.  The  bottom  line  is  that  working  software  is  valued  over  copious  amounts  of 
documentation.  The  question  then  becomes,  given  the  inherent  differences  and  the  continuous 
end-user  involvement,  how  is  an  Agile  business  process  structured? 

Systems  engineering  students  at  George  Mason  University  conducted  research  on  a 
question  posed  by  Boeing.  Specifically,  Boeing  asked  “whether  the  Agile  methodology  can 
adequately  address  the  rigorous  Department  of  Defense  standards  and  needs  for  documentation 
on  mission-critical  software  systems”  (Gu,  Seung,  &  Stephen  ,  2013).  A  literature  review 
revealed  two  different  perspectives  regarding  Agile  documentation.  “Adapted  Agile”  attempts  to 
conform  to  the  traditional  framework,  while  “Aligned  Agile”  seeks  to  implement  a  more  unique 
Agile-style  documentation  approach.  Adapted  Agile  proponents  argue  that  Agile  does  not 
provide  proper  documentation,  especially  for  mission  critical  systems.  A  disadvantage  of  this 
approach  is  that  it  inhibits  the  flexibility  Agile  exists  to  provide.  An  alternative  model  is  Aligned 
Agile,  in  which  the  documentation  is  actually  aligned  with  the  Agile  principles.  This  requires  an 
inherently  different  management  style  relative  to  traditional  DoD  programs.  The  concern  with 
the  second  model  is  that  this  approach  may  be  unacceptable  for  higher  visibility,  higher  cost 
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DoD  programs  like  nuclear  operations.  From  a  documentation  perspective,  the  differences  in  the 
two  approaches  are  illustrated  in  Figure  7  below  (Gu,  Seung,  &  Stephen  ,  2013). 
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Figure  7:  Adapted  versus  Aligned  Agile  Documentation  (Gu,  Seung,  &  Stephen  ,2013) 


In  an  effort  to  find  a  middle  ground  specific  to  DoD  mission  critical  systems,  the  research 
group  offered  a  hybrid  model  that  provides  more  documentation  than  Agile  typically  advocates, 
but  offers  the  flexibility  inherent  in  the  Agile  model.  This  model  is  known  as  the  Scrum  Agile 
Document  Development  Framework  and  is  illustrated  in  Figure  8  below.  Rather  than  specify  the 
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exact  documentation  that  should  be  provided,  this  model  allows  for  additional  documentation 
requirements  based  on  the  specific  project.  For  example,  Acquisition  Category  (ACAT)  I 
programs,  with  an  acquisition  cost  of  over  $480M  (Defense  Acquisition  University,  2015), 
require  additional  documentation  given  the  size  of  the  effort.  As  illustrated  below,  a  key  tenant 
of  this  model  is  the  Definition  of  Done.  It  is  vital  that  the  program  office  comprehensively 
address  required  documentation  for  the  specific  program  so  that  it  can  be  entered  into  the  product 
backlog,  tracked  and  accomplished.  Instead  of  starting  with  the  traditional  documentation 
requirements,  and  trying  to  tailor  out  unnecessary  documentation,  perhaps  it  is  beneficial  to  start 
with  the  Agile  list  and  add  other  documentation  as  required. 


Figure  8:  Scrum  Agile  Document  Development  Framework  (Gu,  Seung,  &  Stephen  ,2013) 

The  figures  above  illustrate  examples  of  different  Agile  documentation  methodologies. 
Some  of  the  reviews  mentioned  in  the  Agile  process  above  may  not  necessarily  be  formal, 
documented  review,  but  rather  they  may  occur  naturally  based  on  the  continuous  interaction 
inherent  in  Agile  processes.  The  documentation  required  will  be  specific  to  each  program,  but 
there  are  certain  documents  that  can  serve  as  a  baseline.  These  documents  include  a  product 
vision,  product  backlog,  sprint  backlog,  burndown  chart,  development  team  velocity,  and 
definition  of  done  for  each  of  the  user  stories  (Rubin,  2013).  This  is  not  an  authoritative  list,  but 
rather  a  guide  to  developing  a  business  process  that  offers  flexibility  and  allows  for  the  effective 
implementation  of  Agile  processes.  Aside  from  training  and  business  process  reengineering, 
another  area  of  implementation  is  the  contracting  guidance  necessary  to  support  Agile  projects. 
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Contracting  for  Agile 


According  to  a  draft  Software  Engineering  Institute  report,  DoD  program  managers  and 
engineer  staff  stated  that  implementing  contracts  supporting  Agile  is  challenging  (Wrubel  & 
Gross,  2015).  The  main  reason  is  because  contracting  for  Agile  is  inherently  different  than 
writing  a  contract  under  the  traditional  waterfall  approach,  and  contracting  officers  lack  the 
training  and  experience  necessary  to  deal  with  this  change.  Under  the  old  process,  the  program 
manager  gave  requirements  to  the  contracting  officer  at  the  very  beginning  of  the  development 
effort,  and  it  was  assumed  that  these  requirements  would  not  change.  If  changes  needed  to  be 
made,  they  would  undergo  a  rigorous  Engineering  Change  Proposal  (ECP),  which  would 
necessitate  a  contract  modification.  Modifications,  especially  when  changing  requirements 
significantly,  lead  to  questions  about  scope  and  could  result  in  having  to  cancel  a  contract  and  re¬ 
compete  it.  In  contrast,  Agile  projects  start  with  a  set  of  broad  requirements,  and  changes  in 
these  are  expected  throughout  the  life-cycle  of  the  project.  A  starting  point  for  understanding  the 
differences  in  these  methodologies,  from  a  contractual  perspective,  is  the  TechFAR  (Technical 
Federal  Acquisition  Regulation).  The  White  House  published  the  TechFAR  and  its  primary 
purpose  is  to  offer  tips,  guidance,  sample  language,  etc.,  to  assist  in  the  implementation  of 
contracts  that  support  Agile  (The  TechFAR  Handbook  for  Procuring  Digital  Services  Using 
Agile  Processes,  2014).  Figure  9  below,  from  the  TechFAR,  illustrates  the  differences  in  pre¬ 
award  and  post-award  for  both  traditional  and  Agile  development.  The  TechFAR  serves  as  a 
good  starting  point  for  contracting  officers  to  begin  to  understand  the  differences  between 
traditional  and  Agile  development  from  a  contracting  perspective  and  it  contains  draft  language 
that  serves  as  an  example  of  how  to  contract  for  an  Agile  process. 


Figure  9:  Events  in  Pre-Award  and  Post-Award  (The  TechFAR  Handbook  for  Procuring  Digital 

Services  Using  Agile  Processes,  2014) 
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A  Software  Engineering  Institute  (SEI)  report  noted  common  misconceptions  regarding 
Agile,  which  include  that  Agile  does  not  produce  enough  documentation,  it  does  not  offer 
enough  insight,  requirements  are  nebulous  and  inherently  risky,  and  finally  that  iterations  will 
require  additional  contracting  overhead  (Wrubel  &  Gross,  2015).  The  first  two  can  be  addressed 
in  concert.  Agile  actually  offers  much  greater  insight  into  the  development  effort  relative  to  the 
traditional  method,  even  though  most  Agile  processes  do  not  contain  traditional  milestones 
(Preliminary  Design  Review,  Critical  Design  Review).  It  relies  upon  constant  communication 
with  the  Product  Owner,  frequent  release  cycles  of  functional  software,  early  testing  and 
integration,  as  well  as  continuous  reprioritization  of  requirements.  As  a  result,  the  government 
office  obtains  a  significant  amount  of  insight  into  the  development  process  and  progress  towards 
completion,  which  is  illustrated  in  Figure  10  below.  The  lack  of  insight  in  the  traditional  model 
necessitated  copious  amounts  of  documentation.  By  contrast,  given  the  numerous  opportunities 
for  quality  checks  and  feedback,  Agile  focuses  on  documentation  that  is  necessary  to  provide 
continuity  and  competition  in  the  future.  When  deciding  to  request  additional  documentation 
from  a  contractor  executing  Agile,  it  is  imperative  that  both  the  contracting  officer  and  the 
program  manager  ensure  the  documentation  is  absolutely  necessary,  sufficient  or  whether  it  is  a 
statutory  or  regulatory  requirement  (Wrubel  &  Gross,  2015). 


Figure  10:  Many  Quality  Touch-Points  in  Agile  Development  (Wrubel  &  Gross,  2015) 

The  next  critique  is  that  requirements  are  poorly  defined  and  inherently  riskier  than  the 
traditional  model.  As  discussed  earlier,  a  major  difference  in  these  approaches  is  that  traditional 
methods  require  certainty  in  requirements  at  the  beginning  of  the  project,  despite  the  fact 
requirements  continually  evolve  and  become  more  refined  throughout  project  lifecycle. 

However,  this  does  not  equate  to  less  risk  in  the  traditional  model.  One  of  the  major  reasons  cited 
by  GAO  for  program  cost  and  schedule  growth  in  traditional  programs,  is  a  lack  of 
understanding  or  growth  in  the  initial  requirements  set  (Government  Accountability  Office 
Military,  2015).  By  contrast,  Agile  models  focus  on  a  big-picture  understanding  of  the 
requirements  early,  while  refining  the  specific  requirements  as  the  team  approaches  the  iterations 
or  sprints  within  which  they  are  contained.  This  prevents  the  government  from  spending  money 
that  focuses  on  all  of  the  requirements  at  once,  and  rather  on  those  that  are  at  the  top  of  the 
priority  list. 


16 


The  key  to  minimizing  contract  overhead  when  executing  Agile  is  to  ensure  the  contract 
is  structured  appropriately.  In  a  traditional  project,  a  contract  modification  is  typically  required  to 
change  requirements;  this  change  process  is  described  as  “expensive  and  time-consuming” 
(Wrubel  &  Gross,  2015).  Contracts  should  include  infonnation  that  sets  the  expectations  for  the 
development  team  as  well  as  the  program  office.  These  include  identification  of  the  Agile 
process  to  be  used,  specification  of  the  different  roles  (Product  Owner,  etc.)  and  the  relative  level 
of  involvement,  specification  of  the  requirements  management  process,  identification  of  the 
expected  level  of  interaction  between  the  contractor  and  the  customer,  and  identification  of 
release  schedules  (Northern,  Mayfield,  Benito,  &  Casagni,  2010).  In  Agile,  changes  to 
requirements  are  expected  and  occur  quite  frequently.  The  key  to  this  flexibility  is  in  the 
contracting  strategy  chosen. 

Finding  exactly  which  contracting  strategy  provides  the  flexibility  needed  for  Agile 
remains  a  crucial  task.  Ms.  Kelly  Goshorn  noted  that  she  can  tell  if  an  Agile  contracting  effort 
will  be  successful  by  which  contract  strategy  they  have  in  place  (2015).  Throughout  her  tenure  as 
the  PEX  PM,  she  used  the  Time  and  Materials  contract  mentioned  earlier,  which  she  considers 
absolutely  critical  for  the  success  of  her  effort.  This  contract  type  gives  the  program  office  the 
same  flexibility  it  provides  to  acquisitions  officers  to  change  and  alter  requirements  without 
conducting  a  time-consuming  contract  modification.  This  contract  is  not  preferred  by  contracting 
officers  because  the  government  bears  a  significant  burden  of  the  risk,  and  there  is  little  incentive 
for  the  contractor  to  keep  costs  down.  Ms.  Goshorn  explained  that  this  methodology  relies  upon 
a  high  level  of  trust  between  the  contractor  and  development  team  as  well  as  continuous 
monitoring  of  the  development  effort  (Kelly  Goshorn,  personal  communication,  August  18, 
2015). 


Initially,  AFSOC’s  Lead-Off-Hitter  (LOH)  program  started  off  with  a  Level  of  Effort 
(LOE)  contract,  but  recently  switched  to  a  Cost  Plus  Fixed  Fee  (CPFF)  construct.  The  program 
office  decided  to  switch  strategies  because  LOE  is  difficult  to  measure  performance  and  manage. 
Additionally,  minimal  control  over  deliverables  exists.  The  LOH  team  actually  uses  six -month 
schedule  driven  releases.  The  program  office  maintains  a  “Capability  Candidates”  list,  which 
currently  contains  around  88  requirements,  and  end  users  may  add  to  anytime.  Every  six  months, 
the  Program  Office,  the  contractor,  and  the  end  users  meet  to  review  the  requirements,  arrive  at  a 
“definition  of  done”  for  each  requirement,  and  prioritize  everything  on  the  list.  Based  on 
previous  releases,  the  program  office  knows  that  the  contractor  can  complete  two  Large,  two 
Medium,  and  one  Small  requirement  every  six  months.  Thus,  the  government  office  decides 
which  task  will  be  in  the  next  six  month  release,  based  on  size  and  prioritization,  and  the 
development  team  begins  work.  The  contracting  officer  issues  a  letter  to  the  development  team, 
stating  that  these  are  the  requirements  in  the  next  release,  and  no  modification  is  required.  The 
schedule  remains  the  driving  force  behind  delivery;  if  one  of  the  requirements  is  not  complete, 
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the  release  will  occur  as  scheduled  with  the  four  that  are  completed.  This  contract  is  structured 
such  that  it  provides  the  same  flexibility  as  a  Time  and  Materials,  and  the  risks  are  more  evenly 
shared  between  the  contractor  and  government. 

None  of  the  programs  interviewed  for  this  paper  used  a  Finn-Fixed  Price  (FFP)  contract; 
however,  the  program  managers  did  offer  insight  into  the  viability  of  this  approach.  The 
traditional  FFP  model  with  fixed  scope  is  not  a  good  fit  given  the  evolving  nature  of 
requirements  in  an  Agile  process.  One  approach  that  might  be  conducive  to  FFP  is  a  very  short 
Agile  contract,  with  regards  to  time.  For  example,  a  program  office  could  solicit  proposals  for  a 
six  month  to  one  year,  or  enough  time  for  one  software  release  effort  that  focuses  on  a  very  small 
set  of  user  stories  within  the  Product  Backlog.  The  PMO  would  need  to  ensure  that  the  user 
stories  selected  were  specific  enough,  even  down  to  the  feature  level,  to  allow  for  the  contractors 
to  provide  a  reliable  estimate.  This  strategy  would  hinge  upon  the  contracting  officer’s  ability  to 
award  the  short  contracts  relatively  quickly  given  the  timeframe  of  each  contract.  According  to  a 
2009  Standish  Group  report,  small  projects  have  a  higher  probability  of  success,  stating 
“software  projects  with  labor  costs  less  than  $750K  have  a  71%  probability  of  success,  those 
costing  between  $750K  to  $3M  only  have  a  19%  chance  of  success,  and  for  projects  over  $10M, 
success  rapidly  falls  to  an  abysmal  2%”(Northern,  Mayfield,  Benito,  &  Casagni,  2010).  The 
rationale  above  can  be  applied  to  all  of  the  contracting  strategies  mentioned.  Rather  than 
awarding  large,  multi-year  contracts,  the  software  development  community  should  shift  to  more 
frequent  awards  of  smaller  efforts. 

One  final  topic  deals  with  how  to  evaluate  contractors  submitting  proposals  under  this 
new  process.  A  necessary  factor  mentioned  by  programs  using  Agile  today  is  the  contractor’s 
prior  experience  with  Agile  processes.  As  discussed  earlier,  in  order  for  Agile  to  be  effective  it 
must  be  understood  and  executed  effectively.  Every  Agile  methodology  is  accompanied  by 
various  documentation  that  is  used  to  track  progress.  In  Scrum,  for  example,  teams  typically  use 
a  product  backlog,  sprint  backlog,  and  a  burndown  chart  as  a  minimum.  If  a  contractor  is 
proposing  an  Agile  methodology,  it  would  be  beneficial  for  the  government  to  request 
documents  like  those  listed  above  to  ensure  the  contractor  is  experienced  in  the  methodology 
they  are  proposing.  Another  benefit  of  Agile  methodologies  are  associated  metrics  that  are 
produced.  Generally  speaking,  all  of  the  user  stories  receive  a  relative  size  estimate,  and 
development  teams  have  a  velocity,  or  speed  at  which  they  accomplish  the  work.  Although 
velocities  are  subject  to  change  seeing  as  each  project  is  different,  the  contractor  could  provide 
metrics  from  an  analogous  workload  for  comparison.  Government  PMOs  could  require  an  oral 
evaluation  during  which  a  contractor  actually  completes  their  first  sprint,  set  at  a  week,  based  on 
the  infonnation  provided  in  their  proposal.  After  the  completion  of  the  first  sprint,  teams  would 
show  their  results  to  the  PMO,  and  present  the  supporting  agile  methodology  documentation. 
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This  would  allow  the  PM  insight  into  the  development  team’s  understanding  of  the  requirements, 
while  illustrating  the  team’s  knowledge  of  the  Agile  methodology  being  proposed. 

Contracting  for  Agile  is  inherently  different  than  contracting  for  a  traditional  waterfall 
project.  It  is  imperative  that  contracting  officers  be  aware  of  Agile’s  common  misconceptions, 
understand  the  options  that  exist  for  Agile  contracting,  and  know  the  options  for  evaluation.  The 
key  to  an  Agile  contract  is  providing  the  flexibility  that  Agile  needs  to  be  effective.  As  the 
examples  above  illustrate,  there  is  no  one  right  contracting  strategy,  but  rather  the  path  chosen  is 
dependent  upon  each  specific  situation.  Further  research  is  needed  in  this  area,  and  best  practices 
should  be  compiled  within  the  acquisitions  community  to  serve  as  a  model  for  other  programs  to 
emulate. 


Summary  &  Recommendations 

Full-scale  Agile  adoption  requires  the  Air  Force  to  focus  on  providing  training  and 
coaches/mentors  to  the  entire  program  management  office,  update  business  processes  to  only 
necessitate  documentation  that  is  actually  needed  and  provide  contracting  guidance  that  ensures 
the  contract  strategy  provides  the  flexibility  Agile  requires.  In  the  short  term,  training  can  be 
achieved  through  commercial  institutions  such  as  PMI.  In  the  long  term,  DAU  should  focus  on 
developing  Agile  courseware,  resident  courses,  and  potentially  a  master’s-level  curriculum  that 
all  cater  to  the  unique  environment  of  Air  Force  programs.  Agile  coaches  and  mentors  should  be 
provided  for  individual  programs,  given  the  uniqueness  of  each  program  and  the  multitude  of 
Agile  methodologies,  to  deliver  tailored  recommendations.  The  Air  Force  needs  to  provide  a 
business  process  that  supports  Agile  principles,  one  that  values  the  working  product  over 
comprehensive  documentation.  Guidance  could  be  provided  to  offer  program  offices  a 
documentation  model  to  follow  and  build  on  as  needed,  as  opposed  to  every  program  office 
individually  tailoring  the  traditional  documentation  model  to  fit  their  specific  needs.  The  final 
recommendation  is  to  offer  contracting  guidance  to  ensure  Agile  contracts  are  provided  the 
flexibility  that  Agile  requires  to  be  successful.  The  Air  Force  should  also  begin  to  gather  Agile 
success  stories  to  share  with  the  larger  acquisitions  community.  Although  each  program  is 
unique,  these  successes  could  offer  templates,  as  well  as  contracting  language  and  metrics, 
among  other  lessons  learned,  that  could  ease  the  transition  for  other  programs,  or  serve  as  best 
practices.  In  this  fiscally  constrained  environment,  the  Air  Force  does  not  have  the  time  nor  the 
resources  for  every  program  to  reinvent  the  wheel  for  how  to  implement  Agile. 

The  changes  elucidated  in  this  paper  will  require  significant  buy-in  from  senior 
leadership,  the  acquisition  community,  and  the  end-user  community.  Senior  leadership  should  be 
pushing  their  program  managers  to  adopt  Agile  strategies  over  the  traditional  methodology  given 
the  numerous  benefits  elaborated  upon  above.  A  culture  change  that  focuses  on  delivering 
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functional  capability  within  months  is  absolutely  critical.  Numerous  reports  and  failures  of 
programs  illustrate  that  the  current  waterfall  methodology  is  antiquated  and  is  unable  to  meet  the 
demands  of  a  changing  threat  environment.  The  recent  updates  to  the  DoDI  5000.02  are  a 
positive  step  in  the  right  direction,  but  embodying  the  Agile  principles  is  more  than  just 
incremental  development;  Agile  is  undoubtedly  the  acquisitions  process  for  the  future.  Programs 
within  the  DoD  are  implementing  Agile  methodologies  and  realizing  success  relative  to  the  old 
processes.  The  Air  Force  must  change  Agile  from  the  exception  to  the  norm  for  software 
development  programs.  Full-scale  adoption  of  Agile  processes  requires  implementation  in 
training,  business  process  re-engineering,  and  contracting  guidance.  As  suggested  by  MITRE, 
partial  implementation  will  not  lead  to  successful  results.  Agile  implemented  correctly  has  the 
ability  to  deliver  the  most  needed  functional  capability  on  budget  and  on  schedule.  The  reality  is 
that  today’s  warfighter  requires  a  functional  solution  that  meets  75%  of  the  requirements  within 
months;  the  current  acquisitions  process  does  not  meet  this  need.  Agile  offers  a  process  that  can 
accomplish  this  feat  and  has  already  demonstrated  success  in  both  the  private  and  public  sectors 
(Lapham,  et  ah,  2011).  Implementing  Agile  into  the  culture  of  Air  Force  acquisitions  will  help 
propel  the  Air  Force  into  the  future,  ensuring  innovation  and  flexibility  as  we  move  further  into 
the  twenty-first  century. 
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